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Test  and  evaluation  (T&E)  of  geographically  dispersed  integrated  systems  are  severely 
constrained  by  cost,  range  safety  restrictions,  and  ability  to  test  while  in  an  operational  state. 
The  Missile  Defense  Agency  has  embarked  on  a  hardware-in-the-loop  (HWIL)  framework 
development  that  has  the  capability  to  characterize  the  performance  of  the  Ballistic  Missile 
Defense  System  by  integrating  the  operational  software  in  a  distributed  laboratory  architecture. 
The  HWIL  framework  is  also  intended  to  test  the  operational  assets  in  their  fielded 
configuration  and  location.  As  more  advanced  radar  discrimination  algorithms  are  developed, 
testing  these  algorithms  and  determining  the  impact  on  the  system  performance  becomes 
increasingly  more  difficult.  The  ability  to  stimulate  radar  signal  processors  with  synthetic 
signatures  has  also  advanced  over  the  last  few  years,  thus  enabling  greater  opportunity  for 
testing.  The  integration  of  separate  defense  programs,  and  thus  independently  developed 
HWILs,  has  been  a  concern  for  the  agency.  The  development  of  the  Ballistic  Missile  Defense 
System  HWIL  will  provide  the  agency  with  a  unified  architecture  across  all  Missile  Defense 
Agency  programs,  allowing  consistent  threat  and  environmental  effects  across  all  systems. 

Key  words:  accreditation;  advanced  test  facilities;  complex  operational  systems;  integrated 
network;  realistic  mission  environments;  simulator/stimulator  testing  labs;  verification  & 
validation. 


Using  the  Ballistic  Missile  Defense 
System  (BMDS)  as  an  example,  this 
article  articulates  the  Missile  Defense 
Agency’s  (MDA)  hardware-in-the- 
loop  (HWIL)  framework  design  and 
development  for  testing  the  BMDS.  This  framework 
will  allow  MDA  to  establish  a  degree  of  confidence  in 
the  expected  performance  of  a  very  complex  opera¬ 
tional  system  that  cannot  be  evaluated  by  conventional 
tests.  The  inherent  difficulty  in  executing  an  opera¬ 
tional  test  in  the  conventional  sense  presents  the 
Operational  Test  and  Missile  Defense  Agencies  with 
challenges  to  field  such  a  complex  system. 

This  article  examines  the  benefits  and  challenges  of 
implementing  a  distributed  HWIL  framework  and 
articulates  areas  that  are  critical  in  design,  implemen¬ 
tation,  and  execution  of  the  BMDS  HWIL.  In 
addition,  the  framework  test  and  control  functions, 
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communication  architecture,  and  interface  require¬ 
ments  are  discussed.  Topics  include 

•  BMDS  components 

•  BMDS  HWIL  fidelity  requirements 

•  Challenges  of  distributed  simulation  execution, 
including  data  latency,  data  rates,  and  synchro¬ 
nization 

•  Management  and  coordination  of  complex  test 
requirements 

•  Common  threat  and  environment  for  stimulation 
of  simulation  elements 

•  Methods  for  HWIL  verification,  validation,  and 
accreditation. 

The  ballistic  missile  defense  system 

The  BMDS  Program  is  designed  to  provide 
protection  against  limited  ballistic  missile  attacks 
targeted  at  the  United  States.  The  MDA  mission  is 
to  develop,  test,  and  field  this  missile  defense  system. 
Using  complementary  interceptors;  land-,  sea-,  air-, 
and  space-based  sensors;  and  battle  management 


29(1)  •  March  2008  23 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

2QQg  2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2008  to  00-00-2008 

4.  TITLE  AND  SUBTITLE 

Design  of  the  Ballistic  Missile  Defense  System  Hardware-in-the-Loop 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Army  Aviation  &  Missile  Command, Systems  Simulation  &  Development 
Directorate, Huntsville, AL, 35898 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

_ _ _  ABSTRACT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  OS 

unclassified  unclassified  unclassified  Report  (SAR) 

6 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Buford,  Pate,  8c  Vatz 


command  and  control  systems,  the  planned  missile 
defense  system  will  be  able  to  engage  all  classes  and 
ranges  of  ballistic  missile  threats.  All  ballistic  missiles 
share  a  fundamental  characteristic — they  follow  a 
trajectory,  which  includes  three  phases:  boost,  mid¬ 
course,  and  terminal.  By  fielding  a  layered  defense 
system  and  attacking  the  missile  in  all  phases  of  flight, 
MDA  can  exploit  opportunities  to  increase  the 
effectiveness  of  missile  defenses  and  complicate  an 
aggressor’s  plans.  The  MDA  has  connected  several  test 
ranges  to  form  the  BMDS  Test  Bed,  which  will  add 
realism  to  ground-  and  sea-based  midcourse  testing  by 
allowing  multiple  engagements  and  different  trajecto¬ 
ries  and  adding  additional  intercept  areas.  The  BMDS 
Test  Bed  also  includes  boost  and  terminal  segment 
tests,  which  will  demonstrate  the  viability  of  the 
layered  missile  defense  concept. 

The  potential  boost-phase  defense  elements  are 
high-power  Air-Borne  Lasers  and  kinetic  energy 
systems.  The  primary  elements  in  the  midcourse  phase 
are  the  Aegis  Ballistic  Missile  Defense  and  the 
Ground-based  Midcourse  Defense  (GMD).  The 
terminal  elements  are  the  Theater  High  Altitude  Area 
Defense  (THAAD)  and  the  Patriot  Advanced  Capa¬ 
bility  3  (PAC-3).  Other  elements  include  the  exper¬ 
imental  Space  Tracking  and  Surveillance  System  along 
with  its  strategic  and  theater  mission  controller,  the 
Command  &  Control  Battle  Manager  and  Commu¬ 
nication  system,  and  other  agency  experimental  and 
operational  sensors. 

The  test  and  evaluation  challenge 

Classical  test  and  evaluation  (T&E)  of  a  new 
weapon  system  entails  repeated  live  “firings”  by  forces 
that  would  be  employing  the  system  against  the 
expected  threats  in  an  environment  similar,  if  not 
identical,  to  the  expected  battle  space.  Although  the 
BMDS  Test  Bed  provides  for  more  realistic  opera¬ 
tional  testing  and  capability  assessments,  only  a  limited 
number  of  flight  tests  will  be  conducted.  In  support  of 
system  assessment  activities,  the  T8cE  community  will 
use  flight  test,  digital  simulation,  and  HWIL  simula¬ 
tion  data. 

The  BMDS  HWIL  framework  provides  a  means  to 
test  the  BMDS  operational  software  in  a  controlled 
laboratory  environment.  The  HWIL  framework  is  also 
intended  to  test  the  operational  assets  at  their  fielded 
sites  and  host  country.  As  new  advanced  radar 
algorithms  are  developed,  the  need  to  inject  threat 
stimuli  directly  into  the  signal  processor  hardware 
increases.  As  much  as  possible,  the  architecture 
incorporates  the  component  operational  processing 
hardware  and  software  that  will  be  used  in  the  field, 


implementing  the  “Test  What  You  Fly,  Fly  What  You 
Test”  paradigm. 

As  the  BMDS  Block  upgrades  are  developed,  the 
impact  on  system-level  performance  must  be  deter¬ 
mined.  The  HWIL  framework  will  allow  MDA 
management  to  evaluate  the  upgrades  before  fielding. 

The  MDA  is  requiring  the  BMDS  HWIL  to 
support  BMDS  system-level  performance-based  as¬ 
sessments  and  support  BMDS  system-level  concurrent 
test  training  and  operations  functions.  The  HWIL 
framework  will  allow  simultaneous  execution  of 
engagement  sequence  groups;  testing  both  theater 
and  strategic  assets.  MDA  can  use  test  data  to  assess 
interoperability  of  MDA  elements,  demonstrate  the 
Command  &  Control  Battle  Manager  and  Commu¬ 
nication  system  capability  to  control  and  manage 
BMDS  communication  networks,  sensor  management, 
and  display  situational  awareness  to  the  warfighter. 

The  Operational  Test  Agency  also  uses  this  test  data 
to  characterize  BMDS  operational  capability,  which 
includes  threat  detection,  tracking,  discrimination, 
engage,  intercept,  and  destroy.  Other  objectives 
include  characterization  of  information  exchange 
capabilities  among  BMDS  elements.  The  warfighter 
additionally  wants  to  verify  courses  of  action,  tactics, 
techniques,  and  procedures. 

Benefits  to  HWIL  testing 

With  the  complexity  of  the  BMDS,  integrating 
multiple  systems  into  a  joint  fighting  force  is  a 
challenge.  Each  element  is  a  completely  different 
acquisition  and  each  has  somewhat  different  require¬ 
ments.  Being  separate,  each  element  does  not  know 
exactly  what  dependencies  and  needs  it  requires  for 
interoperability  with  the  other  elements.  Independent 
testing  and  verification  of  the  elements  does  not 
necessarily  fully  verify  the  BMDS  or  fully  assess  the 
system  capabilities.  If,  for  instance,  the  boost-phase 
elements  cannot  destroy  the  threat,  their  tracking  data 
could  be  used  to  enable  the  midcourse  battle-manager 
to  use  earlier  and  more  accurate  data  to  cue  the 
midcourse  element  radars.  The  benefits  of  the  BMDS 
HWIL  are  to  help  in  flight  test  planning,  interoper¬ 
ability,  and  performance  assessment. 

Flight  test  planning  includes  development  of  flight 
test  concept  of  operations,  timeline  analysis  for  the 
mission  director,  determination  of  when  to  filter  or 
include  range  radar  track  reports,  evaluation  of  the 
exclusion  of  test  range  assets,  pre-mission  testing, 
verification  of  element  interfaces,  predicting  the 
probability  of  mission  success,  and  testing  of  off- 
nominal  excursions. 

The  BMDS  HWIL  may  also  be  instrumental  in  the 
design  and  development  of  the  BMDS  Battle  Manag- 
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er,  which  will  have  to  interface  with  all  element  battle 
management  systems.  Areas  of  interest  include  mes¬ 
sage  translation,  message  traffic  analysis,  situational 
awareness,  allocation  of  interceptors,  track  correlation, 
search  cueing,  drop  track  reasoning,  estimates  of  sensor 
covariance,  and  hand-over  strategies  between  sensors 
of  different  elements  during  different  engagement 
phases. 

The  most  critical  benefit  is  determining  system 
capability  and  testing  of  block  upgrades.  The  results  of 
HWIL  testing  can  be  used  to  demonstrate  and  verify 
that  system  requirements  are  met.  Analysis  efforts 
include  system  capability  assessment,  kill  vehicle  and 
sensor  acquisition,  tracking  and  discrimination,  and 
system  battle-space  evaluation. 

HWIL  description 

This  article  provides  a  construct  for  implementation 
of  a  BMDS  HWIL  and  is  defined  to  include  as  much 
as  possible  the  tactical  hardware  and  software.  HWIL 
facilities  consist  of  space-based  and  radar  sensors, 
interceptors,  and  battle  management  and  communica¬ 
tions.  Obviously  the  radar  antenna  and  the  interceptor 
booster  cannot  be  implemented  in  their  entirety. 
Typically,  the  radar  HWIL  consists  of  the  data 
processors  and,  in  some  instances,  the  signal  proces¬ 
sors.  The  interceptor  HWIL  usually  consists  of  the 
data  processors,  which  execute  the  guidance  software 
and  the  software  utilized  to  process  the  seeker  imagery 
and  determine  the  interceptor’s  acquisition,  tracking, 
and  discrimination  performance.  Typically  the  Battle 
Manager  is  represented  by  the  actual  tactical  hardware 
and  software,  with  the  communication  interfaces  and 
simulated  delays  and  timing. 

The  BMDS  HWIL  will  integrate  laboratory 
facilities  in  locations  across  the  United  States  and 
integrate  the  fielded  operational  assets,  including  those 
in  other  countries  and  at  sea.  The  BMDS  HWIL  will 
contain  a  network  to  transmit  simulation  truth  data  to 
the  elements;  a  tactical  communication  network  is  also 
available  to  exercise  and  evaluate  the  real  communica¬ 
tion  between  elements.  The  simulation  network  uses 
the  simulation  protocol  messages,  while  the  tactical 
network  uses  satellite  and  fiber-optic  links,  with  a 
variety  of  tactical  message  types. 

The  development  of  the  BMDS  HWIL  framework 
will  provide  the  agency  with  a  unified  architecture 
across  all  MDA  programs,  allowing  consistent  hard¬ 
ware,  environment,  and  threat  stimulation.  Common¬ 
ality  is  needed  in  order  to  reduce  risk.  The  benefits  to 
achieving  commonality  in  the  target  generator  include: 

•  Ensuring  confidence  and  control  of  target  data — 
“Single  Source  of  Models.” 


•  Ensuring  consistent  target  representation  across 
multiple  elements — “ALL  right  or  ALL  wrong.” 

•  Minimizing  the  difference  in  performance  be¬ 
tween  elements — “Level  Playing  Field.” 

•  Reducing  development/modification  cost  and 
schedule — “One  Time  Fix.” 

•  Reducing  cost  and  schedule  for  element  project 
offices  (provides  elements  with  HW/SW  to  drive 
stand-alone  element  testing/verification). 

•  Reducing  target  6c  environmental  model  verifi¬ 
cation  6c  validation  (V6cV)  cost  and  schedule. 

•  Maximizing  reuse  of  target  development  efforts 
and  code. 

•  Reducing  risk  of  interpretation. 

•  Maximizing  configuration  control. 

•  Providing  linkage  and  heritage  between  elements. 

Depending  on  whether  the  test  is  for  interoperability 

or  performance  verification  significantly  drives  the 
fidelity  and  commonality  of  the  target  generator. 

HWIL  framework.  The  fidelity  of  the  simulation 
representations  can  vary  across  different  programs; 
however,  the  BMDS  system  engineer  and  integrator 
must  determine  the  fidelity  of  the  configuration 
needed  based  on  the  requirements  and  intended  use 
of  the  simulation  output  data. 

The  element  representations  should  at  a  minimum 
have  the  operational  software  integrated  into  the 
simulation  or  hosted  on  the  actual  tactical  data 
processor  hardware.  In  addition,  the  signal  processor 
could  be  added,  along  with  the  missile  HWIL,  and  in- 
band  injection  of  scenes  to  the  sensor. 

The  basic  BMDS  HWIL  architecture  will  consist  of 
the  test,  execution,  and  control  (TEC)  module,  the 
Test  Interface  Unit,  and  the  element  HWIL  repre¬ 
sentations. 

Test,  execution,  &  control  (TEC).  The  importance  of 
the  TEC  module  is  to  establish  the  connectivity  and 
determine  the  particular  test  cases  and  setup  required. 
The  TEC  module  must  synchronize  all  participants’ 
simulation  time  and  provide  the  necessary  initialization 
and  start  commands  to  each  representation.  The  TEC 
module  also  provides  updated  interceptor  state  infor¬ 
mation  from  each  element  to  the  other  elements 
participating  in  the  exercise. 

The  TEC  conducts  three  major  functions:  pre¬ 
mission,  mission,  and  post-mission  execution.  In 
general,  the  BMDS  HWIL  pre-mission  TEC  provides 
single  point  control  in  defining  test  cases  and  providing 
the  capability  to  specify  test  simulation  start  time  (past, 
present,  future). 

During  the  actual  test  event  execution,  the  BMDS 
HWIL  mission  TEC  provides  displays  that  summarize 
BMDS  HWIL  framework  and  element  health  and 
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status,  situational  awareness  of  BMDS  elements  under 
test  (element  positions,  sensor  coverage,  and  threat), 
and  framework  and  system  events  for  monitoring. 
BMDS  HWIL  mission  TEC  also  provides  the 
capability  to  monitor  and  display  run-time  test 
integrity  metrics  to  include  framework  and  tactical 
message  traffic,  message  latency,  and  loss. 

After  completion  of  the  test  case,  the  BMDS 
HWIL  post-mission  TEC  provides  the  capability  to 
import  raw  and/or  processed  data  to  a  centralized 
database  management  system.  This  data  will  be 
provided  to  the  MDA  and  Operational  Test  Agency 
(OTA)  communities  for  analysis. 

Test  interface  unit.  Another  critical  piece  of  any 
HWIL  is  the  target  generator  module.  The  test 
interface  unit  comprises  modules  to  generate  threat 
trajectories  and  dynamics,  radar  signatures,  threat 
plume  intensities,  and  interceptor  signatures.  In 
conjunction,  common  environmental  libraries  are 
utilized  to  induce  effects  to  the  signatures.  The 
environmental  effects  include  ionosphere,  earth  limb, 
refraction,  attenuation  due  to  standard  atmosphere, 
and  rain.  Other  celestial  objects  modeled  include 
satellites,  the  sun,  and  the  moon.  Interceptor  debris 
is  also  modeled.  The  resultant  signatures  are  then 
provided  to  the  component  representations. 

As  more  advanced  radar  discrimination  algorithms 
are  developed,  testing  these  algorithms  and  determin¬ 
ing  the  impact  on  the  system  performance  has  become 
increasingly  more  difficult.  The  ability  to  stimulate 
radar  signal  processors  with  synthetic  signatures  has 
also  advanced  over  the  last  few  years,  thus  enabling 
greater  opportunity  for  testing.  The  test  interface  unit 
will  have  the  ability  to  drive  both  the  data  processor 
and  the  signal  processor  to  minimize  the  cost  impacts 
of  replacing  all  element  representations. 

Having  a  distributed,  HWIL  simulation  architecture 
only  amplifies  the  need  for  adequate  timing  analysis. 
Bandwidth  often  limits  the  data  rates  between  facilities 
and  elements.  The  HWIL  system  architectural  engineer 
must  determine  the  data  rates  at  each  level  of  the 
simulation  from  the  TEC,  to  the  target  generator,  to  the 
element  interface,  and  even  the  rates  associated  with 
tactical  communications  between  the  elements.  A  test 
interface  unit  will  be  co-located  with  each  component  to 
minimize  data  latency.  Each  component  will  have  to 
have  an  element-specific  interface  to  incorporate  the 
different  radar  waveforms  and  integration  rates  needed. 

MDA  test  events 

The  MDA  has  embarked  on  a  test  campaign  for 
each  year  and  block  upgrade.  The  campaign  consists  of 
laboratory  testing  and  operational  asset  testing. 


Ground  Test-Integrated  (GTI)  will  be  a  distributed 
laboratory  system-level  test,  utilizing  MDA  element 
HWIL  facilities.  The  purpose  of  the  test  is  to 
demonstrate  the  performance  capability  of  the  BMDS. 
The  GTI  will  provide  data  for  element  and  system- 
level  assessments  by  executing  a  variety  of  scenarios  and 
conditions,  and  evaluating  sequences  of  events  from 
the  BMDS  kill  chain  (e.g.,  detection,  tracking, 
engagement,  etc.). 

Ground  Test-Distributed  (GTD)  will  be  a  distrib¬ 
uted  fielded  system-level  test.  Each  BMDS  element 
has  incorporated  into  the  tactical  operational  software 
the  ability  to  execute  simulated  tests,  similar  to  the 
HWIL  laboratories.  The  major  difference  between  the 
GTD  and  GTI  is  that  the  GTD  will  exercise  the 
tactical  communication  links  from  the  actual  fielded 
locations.  In  general,  the  test  cases  in  the  GTD  are  a 
subset  of  the  GTI.  The  GTD  is  a  progression  of  the 
GTI  testing.  GTD  are  intended  to  double  check  that 
the  performance  of  the  operational  assets  replicate  the 
performance  evaluated  during  the  GTI  test  campaign. 

The  concurrent  test  training  and  operations  concept 
will  capitalize  on  the  GTD  architecture  to  allow  the 
warfighter  the  opportunity  to  train  and  test  on  the 
operational  assets,  while  maintaining  operational  capa¬ 
bility  to  defend  the  nation.  This  concept  will  increase 
the  requirements  on  both  the  HWIL  framework  and 
the  operational  system.  However,  the  benefits  to  the 
warfighter  to  train  while  on  station  will  significantly 
increase  troop  efficiency.  The  crews  will  be  able  to 
evaluate  their  tactics,  techniques,  and  procedures  and 
the  command  structure  communications. 

Evaluation 

The  test  requirements  process  is  a  large  and  complex 
job.  The  challenge  of  writing  good  test  requirements 
can  be  lessened  if  the  flow  down  process  is  used  to 
define  overall  objectives  and  operational  scenarios. 
These  will  flow  down  to  the  system  requirements, 
which  will  flow  down  to  the  subsystem  requirements, 
and  so  on  down  to  the  test  requirements.  Simulta¬ 
neously  while  developing  a  flow  down  process  for  the 
requirements,  each  requirement  must  be  verifiable  and 
able  to  fit  into  specifications.  Good  test  requirements 
will  be  very  specific  and  reflect  the  functionality  of  the 
components  and,  in  turn,  the  system. 

The  primary  objective  of  any  evaluation  activity  is  to 
determine  if  the  test  objectives  and  requirements  have 
been  met.  This  requires  that  any  observed  or  potential 
system  performance  shortfalls  be  identified.  A  com¬ 
prehensive  set  of  system  performance  measurements, 
applied  on  a  per-run  basis  is  used  to  verify  that  system 
performance  is  maintained  within  established  margins. 
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These  margins  define  the  limits  of  system  performance 
relative  to  ensuring  successful  test  implementation. 

During  each  test  case  run,  the  critical  mission 
timeline  and  the  expected  results  for  key  system  events 
will  be  documented  on  the  test  case  run  log  for  each 
test  case.  As  the  test  case  run  is  completed,  the  test 
director  will  indicate  on  the  log  sheet  if  the  key  system 
events  occurred  as  predicted  and  if  the  expected  results 
were  obtained.  All  test  case  anomalies  will  be  recorded 
on  the  test  case  run  log  and  will  be  provided  to  the 
personnel  performing  the  analysis.  After  the  test  case 
runs  are  completed,  a  post-test  analysis  will  be 
performed.  The  analysis  determines  if  the  mission 
objectives  were  met  and  what  the  system  performance 
margins  are  relative  to  the  requirements.  In  the  event 
of  an  anomaly,  further  analysis  will  be  performed  on 
the  test  case  to  determine  the  root  cause  of  the  problem 
and  to  provide  a  resolution.  A  daily  assessment  report 
summarizes  the  information  collected  during  the  post¬ 
test  data  analysis  activities. 

At  the  completion  of  the  test,  the  evaluation  team 
will  produce  a  test  evaluation  report.  The  contents  of 
this  report  will  include  a  comprehensive  evaluation  and 
analysis  of  all  test  objectives  and  test  requirements 
along  with  the  system  level  assessment.  The  results  will 
be  made  available  to  the  BMDS  systems  engineer  who, 
in  turn,  directs  future  development  to  improve 
performance  and  capability. 

HWIL  integration  and  accreditation  process 

There  are  four  phases  in  an  HWIL  integration  and 
accreditation  process.  The  first  phase  is  the  delivery  of 
element  representations  and  their  stand-alone,  checkout 
testing.  During  this  phase,  it  is  the  responsibility  of  the 
element  integrated  process  team  to  deliver  V6cV  data 
certifying  that  the  model  is  a  valid  representation  of  the 
element  within  specified  limitations  and  usage  con¬ 
straints.  The  second  phase  is  the  integration  of  the 
element  representations  into  the  BMDS  HWIL  frame¬ 
work,  in  accordance  with  jointly  defined  integration 
plans.  Both  the  framework  and  element  representations 
verify  the  interface  control  documents  have  been  met. 

The  third  phase  includes  two  distinct  activities:  (a) 
element-to-element  integration  buildup,  and  (b)  test 
readiness.  The  integration  buildup  part  of  this  phase 
includes  testing  each  element  with  system  Battle 
Manager  and  then  testing  with  all  elements  scheduled 
to  participate  in  the  HWIL  configuration.  After 
integration  buildup,  test  readiness  activities  are  conduct¬ 
ed  including  regression  testing,  dry  run  execution,  and 
finally  lock-down  of  the  HWIL  configuration  baseline. 

All  anomalies  found  during  integration,  regression, 
and  engineering  tests  will  be  documented  in  Test 


Incident  Reports  (TIR).  Each  TIR  will  be  isolated  to 
an  operator,  framework,  or  element  issue.  The  TIR  is  a 
management  process  used  for  documenting,  disposi¬ 
tion,  and  tracking  test  incidents  for  future  development 
throughout  the  testing  life  cycle. 

The  output  of  phase  3  is  a  signed  certification  letter 
from  each  participating  element  stating  their  respective 
element  has  been  successfully  integrated  into  the 
HWIL  in  compliance  with  the  Interface  Control 
Document  and  can  support  the  test  objectives  and  test 
requirements.  Collectively,  the  MDA  and  BMDS 
elements  are  executing  an  ongoing  suite  of  V6cV 
activities  to  establish  the  credibility  of  the  element  test 
articles.  Each  element  program  manager  is  responsible 
for  reviewing  the  V6cV  data  and  the  integration  testing 
results,  after  which  caveats  and  limitations  are 
generated.  This  recommendation  is  to  be  delivered  to 
the  accreditation  agent  at  the  Preliminary  Test 
Readiness  Review  (PTRR). 

The  fourth  phase  is  the  accreditation  of  the 
integrated  HWIL  test  configuration.  During  this 
phase,  the  accreditation  agent  produces  an  acceptability 
assessment  and  accreditation  recommendation,  which 
is  provided  to  the  MDA  directors  of  systems 
engineering  and  test  and  evaluation.  The  directors 
evaluate  the  accreditation  recommendation  and  deter¬ 
mine  if  the  configuration  is  ready  for  test.  A  signed 
accreditation  letter  is  then  prepared  and  presented  at 
the  Test  Readiness  Review,  which  allows  the  formal 
start  of  test  execution. 

Inherent  in  this  proposed  accreditation  paradigm  is 
the  execution  with  due  diligence  of  commonly  accepted 
modeling  and  simulation  (M&S)  V6cV  practices. 

Verification  and  validation  (V  &  V) 

Verification  is  the  evidence  of  compliance  with 
requirements  for  a  system  (i.e.,  “Did  I  build  it  right?”). 
Simulation  verification  is  confirmation  that  all  data 
inputs,  logic,  calculations,  and  engineering  representa¬ 
tions  within  the  simulation  accurately  portray  the 
intended  characteristics  and  interactions.  Validation  is 
the  evidence  of  the  system  successfully  achieving  its 
intended  purpose,  or  function  (i.e.,  “Did  I  build  the 
right  thing?”)  Validation  confirms  that  a  simulation 
reflects  real  world  expectations  and  is  generally 
accomplished  by  comparing  simulation  results  to  actual 
flight  test  results  or  other  external  data.  V  6c  V  should 
be  implemented  in  the  initial  stages  of  the  HWIL 
development  and  followed  throughout  its  life  cycle. 

Failure  to  plan  for  proper  V6cV  activities  can  lead  to 
costly  design  and  schedule  ramifications.  A  clear  process 
for  the  flow-down  of  accreditation  needs  into  V6cV  data 
products  and  findings  is  required.  The  specific  V6cV 
activities  identified  for  execution  and  the  resultant  V6cV 
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documentation  is  explicitly  identified  in  a  formal  V&V 
plan.  All  V&V  activities  should  be  selected  for  execution 
with  the  goal  of  satisfying  the  fundamental  data  needed 
to  support  an  accreditation  decision. 

Caveats  and  limitations 

A  key  feature  of  any  accreditation  decision  is  the 
identification  of  the  caveats  and  limitations  associated 
with  the  simulation  configuration.  Caveats  caution  ana¬ 
lysts  on  the  proper  use  of  the  test  data,  while  limitations 
identify  capability  shortfalls  in  the  test  configuration. 
These  caveats  and  limitations  are  linked  to  the  specific 
test  objectives  and  test  requirements  of  a  given  test. 

Accreditation 

In  accordance  with  MDA  policy,  all  core  M&S  will 
be  accredited  to  support  acquisition  decisions.  M&S 
are  abstractions  and  may  not  duplicate  all  actual, 
observed  phenomena;  however,  they  can  provide 
reasonable  approximations.  Based  on  V&V  activities 
and  integration  testing,  an  assessment  is  performed  to 
determine  the  extent  to  which  the  HWIL  configura¬ 
tion  can  meet  specified  test  objectives  and  require¬ 
ments.  Accreditation  is  the  official  determination  that 
the  test  resource  provides  credible  data  that  can  be 
applied  to  meet  the  intended  uses  within  the  stated 
caveats  and  limitations. 

Summary  and  conclusion 

This  article  articulates  how  fundamental  test  objec¬ 
tives  can  be  met  for  a  very  complex  system  of  systems, 
which  cannot  be  evaluated  fully  through  conventional 
developmental  or  operational  tests.  It  examines  the 
benefits  and  challenges  of  implementing  a  distributed 
HWIL  to  support  such  assessments  using  the  BMDS 
as  an  instance.  Areas  that  are  critical  in  design, 
implementation,  and  execution  of  the  BMDS  HWIL 
are  addressed.  Based  on  V&V  activities  and  integration 
testing,  an  accreditation  assessment  is  performed  to 
determine  the  extent  to  which  the  HWIL  configura¬ 
tion  can  meet  specified  test  objectives  and  require¬ 
ments  and  to  establish  a  degree  of  confidence  in  the 
expected  performance.  Q 
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